Networking Issues Labs

1. Basic Network Configuration

Goals

  • Correct network misconfiguration

  • Your server is able to communicate with remote systems

After a recent configuration push from your puppet server, you are unable to reach http://www.opentlc.com/ssh.html.

Investigate and correct this problem.

  1. If you have not already done so, as root on desktop1 and server1 download and uncompress the lab scripts tarball in /root:

    [root@server1 ~]# wget http://www.opentlc.com/download/RTlabscripts.tar.gz
[root@server1 ~]# tar -xf RTlabscripts.tar.gz
    [root@desktop1 ~]# wget http://www.opentlc.com/download/RTlabscripts.tar.gz
[root@desktop1 ~]# tar -xf RTlabscripts.tar.gz

  2. On server1 install and use links to determine if you can get to the Internet.

    [root@server1 ~]# yum -y install links
[root@server1 ~]# links http://www.opentlc.com/ssh.html

    The first time you use links you will get a welcome message; press Enter. You should now see a text version of http://www.opentlc.com/ssh.html. Press q then Enter to exit links.

  3. On server1 install networklab1-1.0-1.noarch.rpm, then attempt to access http://www.opentlc.com/ssh.html again.

    [root@server1 ~]# rpm -i /root/networklab1-1.0-1.noarch.rpm
[root@server1 ~]# links http://www.opentlc.com/ssh.html

  4. What broke

    Users are unable to connect to http://www.opentlc.com/ssh.html from server1. Although this problem may seem familiar, it demonstrates that duplicate problems such as not being able to connect to a server via the network, does not necessarily mean the solutions are the same.

  5. Isolate

    1. A thorough inspection of network configuration files and device settings should identify the problem. Before validating network settings, restart network services and validate the output.

      [root@server1 ~]# ip a
(valid output)
[root@server1 ~]# ip route
(valid output)
[root@server1 ~]# nslookup www.opentlc.com
(fails)

    2. Evaluation of output from the above commands identifies there is no issue with network configuration but the resolver is not working. Investigation of more files is required.

      [root@server1 ~]# cat /etc/resolv.conf

  6. Observations

    1. Further evaluation of configuration files indicates that the problem is an incorrectly configured /etc/resolv.conf file. There appears to be one major configuration error where the nameserver is commented out:

      #nameserver ...

      should read:

      nameserver ...

    2. Fix /etc/resolv.conf.

  7. After fixing resolv.conf the command links http://www.opentlc.com/ssh.html should work, but it still doesn’t.

    A further discovery of settings on server1 should help identify the problem.

  8. Isolate

    Inspect resolver related configuration files.

    [root@server1 ~]# cat /etc/hosts
[root@server1 ~]# cat /etc/resolv.conf
[root@server1 ~]# cat /etc/sysconfig/network
[root@server1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
[root@server1 ~]# cat /etc/nsswitch.conf

  9. Observations

    Inspecting the output of the cat /etc/nsswitch.conf command shows that the hosts: configuration does not include DNS. Add DNS back into the configuration by making the line look like this:

    hosts: files dns

2. Troubleshoot Network Configuration, Round 2

Goals

  • Repair the network following a system crash

  • System is able to successfully boot and access the network

Over the weekend your system crashed and rebooted and various file system recovery tools ran during the reboot. The tools repaired file system problems and performed other clean ups. After the reboot everything seems to be fine except networking fails to start.

Investigate and correct this problem.

This lab will run on the desktop1 system.

  1. Install the networklab2-1.0-1.noarch.rpm rpm on desktop1 and reboot.

    [root@desktop1 ~]# rpm -i networklab2-1.0-1.noarch.rpm
[root@desktop1 ~]# reboot

  2. What broke

    A recent power outage forced several systems to reboot. Reports have come in to the support center that indicate all systems seem to have reinitialized without error, however, desktop1 is inaccessible over the network.

    Additional information was reported concerning recent maintenance on desktop1 prior to the outage. Because no details of the maintenance were provided, a more thorough inspection of configuration files as well as scripts and utilities relative to network services should be considered.

  3. Isolate

    1. Log in to desktop1 as root using the VNC console.

    2. Based on the problem statement, the following sequence of commands suggests one approach to validate network settings and identify the problem.

      [root@desktop1 ~]# ip addr
(no IPv4 address indicates network services have not initialized normally)
[root@desktop1 ~]# systemctl restart network
(output appears to be correct)
[root@desktop1 ~]# ip addr
(again no output, this indicates that the scripts executed
during the last run of the service command did not execute
as expected)

    3. Examine network configuration.

      [root@desktop1 ~]# cat /etc/sysconfig/network
[root@desktop1 ~]# cat /etc/sysconfig/network-scripts/ifcfg-eth0
      BOOTPROTO=static

  4. Repair

    The problem is now evident. Your systems rely on DHCP to get their IP address and BOOTPROTO is set to static.

    Set BOOTPROTO to dhcp and restart networking.

    BOOTPROTO=dhcp
    [root@desktop1 ~]# systemctl restart network

3. Troubleshoot iSCSI Initiator

Goals

  • Correct configuration issue with iSCSI

  • System correctly boots connected to remote iSCSI target storage

Your server1 system was configured to access an iSCSI target published by the 192.168.0.254 host using standard network ports. Another administrator recently made some changes to that system’s disk configuration and now the iSCSI storage will not mount automatically when the system boots.

Investigate and correct this problem.

  1. Create ISCSI target on desktop1 host:

    1. Log in to the desktop1 host as root.

    2. Create iscsi target on desktop1:

      [root@desktop1 ~]# systemctl start target;systemctl enable target
[root@desktop1 ~]# targetcli
/backstores/fileio create desktop1.example.com.disk1 /tmp/fileio 10M write_back=false
/iscsi create iqn.2014-06.com.example:desktop1.example.com
/iscsi/iqn.2014-06.com.example:desktop1.example.com/tpg1/acls create iqn.1994-05.com.redhat:ed2b78dbd5d
/iscsi/iqn.2014-06.com.example:desktop1.example.com/tpg1/luns create /backstores/fileio/desktop1.example.com.disk1
/iscsi/iqn.2014-06.com.example:desktop1.example.com/tpg1/portals create 192.168.0.1
exit

  2. From the desktop1 system, reset the server1 VM.

    [root@desktop1 ~]# lab-resetvm

  3. Once the server1 system has come back, log in to server1 as root via SSH.

    Also, via guacamole, open the console for server1 using virt-manager on desktop1.

  4. Break it (virtual bad system administrator)

    1. Paste the following into server1 (do not fix any errors you see):

      [root@server1 ~]# iqn=`iscsiadm -m discovery -t st -p 192.168.0.1|cut -f2 -d" "`;iscsiadm -m node -T $iqn -p 192.168.0.1 -l;parted -s /dev/sda mklabel msdos mkpart primary 0% 10MB;mkfs /dev/sda1;UUID=`blkid /dev/sda1|cut -f2 -d\"`;echo "UUID=$UUID /mnt ext4 defaults 1 2" >> /etc/fstab

    2. Reboot server1.

      [root@server1 ~]# reboot

  5. Isolate

    From the graphical console for server1 you will see that the system no longer fully boots.

    1. Power off the VM using virt-manager, then power it back on.

    2. Using what you have learned in earlier labs, start the system in single user mode and chroot to the proper location.

    3. Looking at what stopped during boot and what was changed by the bad system administrator, determine what you need to get the system to boot.

  6. Repair

    Network-based disk devices require a _netdev mount option in /etc/fstab. It appears your co-worker thought he was being helpful by changing the option to defaults.

    1. Correct the problem and reboot your machine to confirm the problem is fixed persistently.